Search AI Products and News

With the rapid development of artificial intelligence technology, many companies have launched their own 'AI Agent' systems that can autonomously interact with the environment and complete tasks without human intervention. Companies like Microsoft and Anthropic have been the first to introduce their AI Agents, while industry leader OpenAI has yet to release its own version, and the reasons behind this delay are thought-provoking. According to The Information, OpenAI's delay is related to concerns over 'prompt injection' attacks.

Developer Evan Zhou successfully manipulated Apple's new AI system, Apple Intelligence, in MacOS 15.1 Beta 1 using an attack method called 'prompt injection'. Initially aimed at rewriting text quality, Zhou constructed an attack that overwrites system prompts by leveraging prompt information shared by Reddit users, causing the AI to ignore original instructions and respond with new text. This discovery has garnered widespread attention, although Apple Intelligence...

The Prompt-Guard-86M model released by Meta is designed to defend against prompt injection attacks by restricting large language models from processing inappropriate inputs, thereby protecting system security. However, the model itself also exposes risks of being attacked. Research conducted by Aman Priyanshu found that by adding simple character spacing such as spaces or removing punctuation in the input, the model disregards prior security instructions, achieving an almost 100% success rate for attacks. This finding highlights the importance of AI security, despite Prompt